Skip to content

feat: validation feature inclusion into altimate-code#68

Open
Bharatram-altimate-ai wants to merge 2 commits intomainfrom
validation
Open

feat: validation feature inclusion into altimate-code#68
Bharatram-altimate-ai wants to merge 2 commits intomainfrom
validation

Conversation

@Bharatram-altimate-ai
Copy link
Copy Markdown
Contributor

@Bharatram-altimate-ai Bharatram-altimate-ai commented Mar 6, 2026

What does this PR do?

Adds a validation framework to altimate-code that evaluates AI agent traces against quality criteria (Groundedness, Validity, Coherence, Utility, Tool Validation).

  • Adds /validate skill with SKILL.md and batch_validate.py that calls the Altimate backend API directly via SSE (no Langfuse dependency) for single trace, date range, and session modes
  • Adds validate CLI command (altimate-code validate install/status) that installs the skill to ~/.altimate-code/skills/validate/
  • Adds conversation-logger that auto-logs user/assistant turns to the backend on session idle
  • Wires initConversationLogger() into InstanceBootstrap
  • Auto-installs validate skill after bun run build
  • Fixes build.ts to embed skill assets from the correct path
  • Skill output is written directly to markdown files instead of terminal; user is notified with the file path after each write

Type of change

  • New feature (non-breaking change which adds functionality)

How did you verify your code works?

  • Ran /validate <trace_id> locally and confirmed per-trace markdown files are written to the report directory
  • Verified altimate-code validate install correctly installs the skill
  • Confirmed conversation logger attaches and logs turns on session idle
  • Typecheck passes across all packages

Issue for this PR

N/A — internal feature addition

Checklist

  • Tested locally and confirmed the feature works end-to-end
  • No unrelated changes included in this PR

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants